Device, head mounted display, control method of device and control method of head mounted display

ABSTRACT

A device includes a detection unit that detects states of eyelids of a user, and a control unit that performs operations in response to the states of the eyelids of the user detected by the detection unit.

BACKGROUND

1. Technical Field

The present invention relates to a device, a head mounted display, acontrol method of the device, and a control method of the head mounteddisplay.

2. Related Art

Head mounted displays as displays mounted on heads have been known. Forexample, the head mounted display generates image light representing animage using a liquid crystal display and a light source, guiding thegenerated image light to eyes of a user using a projection system and alight guide plate, and thereby, allows the user to visually recognize avirtual image.

In the head mounted display, it is preferable that the user may not onlyvisually recognize the generated virtual image but also visuallyrecognize input devices such as a mouse, a keyboard, etc. Accordingly, atechnology of providing a sensor that may senses the position of a handof the user (for example, an infrared sensor) and changing the positionof the virtual image to be visually recognized in response to theposition of the hand of the user is known (for example, seeJP-A-6-314331).

Further, in the head mounted display, lower power consumption isrequired. Especially, in the head mounted display driven by a batteryfor improvement of portability, for the longer available time, the lowerpower consumption is further required. In this regard, to reduce thepower consumption, a head mounted display without the necessity of alight emission source is known (for example, see JP-A-7-209600).

JP-A-2009-192583, JP-A-2006-105889, and JP-A-2000-353046 are otherexamples of related art.

However, in the head mounted display in related art, it is necessary forthe user to use a hand at each time of operation and, when display isunnecessary, prior button operation is necessary, and thus, there isroom for improvement in convenience. Note that the problem is notlimited to the head mounted display, but common to all equipment.

Further, in the head mounted display in related art, no light emissionsource is provided, and there is a problem that it may be impossible toview a beautiful image in the environment without sufficient outdoorlight.

SUMMARY

An advantage of some aspects of the invention is to provide a technologyby which a user can operate a device without using a hand. Anotheradvantage of some aspects of the invention is to provide a technology ofsuppressing power consumption without deterioration of quality of adisplayed image in a head mounted display.

Application Example 1

This application example of the invention is directed to a deviceincluding a detection unit that detects states of eyelids of a user, anda control unit that gives instructions in response to the states of theeyelids of the user detected by the detection unit.

In the device, the detection unit detects the states of the eyelids ofthe user and the control unit gives instructions in response to thestates of the eyelids, and thereby, the user can perform operationswithout using hands and the operability may be improved.

Application Example 2

This application example of the invention is directed to a head mounteddisplay including an image display unit having an image light generationpart that generates and outputs image light based on image data and alight guide part that guides the image light to eyes of a user andallowing the user to visually recognize a virtual image, and the deviceaccording to Application Example 1 connected to the image display unit,wherein the control unit transmits the image data to the image displayunit and controls image display by the image display unit, and switchesplural operation modes with different power consumption in response tothe states of the eyelids of the user detected by the detection unit.

In the head mounted display, the detection unit detects the states ofthe eyelids of the user. The control unit switches to the pluraloperation modes with different power consumption in response to thedetected states of the eyelids. Accordingly, the head mounted displayoperates in the appropriate operation mode in response to the states ofthe eyelids of the user, and thereby, may reduce the power consumption.

Application Example 3

This application example of the invention is directed to the headmounted display according to Application Example 2, wherein the controlunit performs power-saving transition as transition to the operationmode with lower power consumption when the detection unit detects theeyelids of the user kept closed in a predetermined time.

If the eye lids of the user are closed in the predetermined time, it isconsidered highly possible that the user is not viewing the image. Inthe head mounted display, the control unit performs power-savingtransition as transition to the operation mode with lower powerconsumption when the detection unit detects the eyelids of the user keptclosed in the predetermined time, and thereby, power consumption may bemade smaller than in the past while reduction of the convenience of theuser is suppressed.

Application Example 4

This application example of the invention is directed to the headmounted display according to Application Example 3, wherein the controlunit performs recovery transition as transition to the operation modewith higher power consumption when the detection unit detects theeyelids of the user opened in a predetermined time after thepower-saving transition.

If the eye lids of the user are opened in the predetermined time, it isconsidered highly possible that the user is viewing the image. In thehead mounted display, the control unit performs recovery transition astransition to the operation mode with higher power consumption when thedetection unit detects the eyelids of the user opened in a predeterminedtime after the power-saving transition, and thereby, the recoverytransition may be promptly performed.

Application Example 5

This application example of the invention is directed to the headmounted display according to Application Example 3 or 4, wherein theimage light generation part includes a light source that outputs lightand an image generation part that converts the light output from thelight source into image light representing an image based on the imagedata, and the control unit reduces the light output from the lightsource at the power-saving transition.

In the head mounted display, the control unit reduces the light outputfrom the light source at the power-saving transition, and thereby, thepower consumption may be reduced.

Application Example 6

This application example of the invention is directed to the headmounted display according to Application Example 5, wherein the lightsource includes a light source for right eye and a light source for lefteye, and the control unit reduces the light output from one of the lightsource for right eye and the light source for left eye at thepower-saving transition.

In the head mounted display, the control unit reduces the light outputfrom one of the light source for right eye and the light source for lefteye at the power-saving transition, and thereby, flexible reduction ofthe power consumption may be realized.

Application Example 7

This application example of the invention is directed to the headmounted display according to Application Example 3 or 4, wherein theimage light generation part includes a light source that outputs lightand an image generation part that converts the light output from thelight source into image light representing an image based on the imagedata, and the control unit stops an operation of the image generationpart at the power-saving transition.

In the head mounted display, the control unit stops the operation of theimage generation part at the power-saving transition, and thereby, thepower consumption may be reduced.

Application Example 8

This application example of the invention is directed to the headmounted display according to Application Example 4, wherein the imagelight generation part includes a light source that outputs light and animage generation part that converts the light output from the lightsource into image light representing an image based on the image data,and the control unit stops an operation of the image generation part atthe power-saving transition and starts the operation of the imagegeneration part again at the recovery transition.

In the head mounted display, the control unit starts the operation ofthe image generation part again at the recovery transition, and thereby,the recovery transition may promptly be performed.

Application Example 9

This application example of the invention is directed to the headmounted display according to Application Example 3 or 4, wherein thecontrol unit includes a functional part that realizes plural functionsand stops a partial operation of the functional part that realizesplural functions at the power-saving transition.

In the head mounted display, the control unit stops the partialoperation of the functional part that realizes plural functions at thepower-saving transition, and thereby, the power consumption may bereduced.

Application Example 10

This application example of the invention is directed to the headmounted display according to Application Example 9, wherein the controlunit further has an operation part and starts the stopped operation ofthe functional part again when the operation part receives a specificoperation after the power-saving transition.

In the head mounted display, the control unit starts the stoppedoperation of the functional part again when the operation part receivesthe specific operation after the power-saving transition, and thereby,even though the detection unit does not detect the states of the eyelidsof the user, the recovery transition may promptly be performed.

Application Example 11

This application example of the invention is directed to the headmounted display according to Application Example 10, wherein the controlunit stores information of the image displayed by the image display unitat power-saving transition and restarts display from the image displayedat the power-saving transition at the recovery transition.

In the head mounted display, the control unit restarts display from theimage displayed at the power-saving transition at the recoverytransition, and thereby, the intended image may be efficiently viewed atthe recovery transition and the convenience may be improved.

Application Example 12

This application example of the invention is directed to the headmounted display according to Application Example 3, wherein the controlunit performs the power-saving transition with much lower powerconsumption when the detection unit detects the eyelids of the user keptclosed in a predetermined time after the power-saving transition.

In the head mounted display, the control unit performs the power-savingtransition with lower power consumption in a stepwise manner in responseto the states of the eyelids of the user. Accordingly, the head mounteddisplay may promptly perform the recovery transition while reducing thepower consumption.

Application Example 13

This application example of the invention is directed to the deviceaccording to Application Example 1, wherein the control unit convertsrespective patterns of the states of the eyelids detected by thedetection unit into input information according to a predetermined rule.

In the device, the detection unit detects the pattern of the states ofthe eyelids and the control unit converts the instruction assigned tothe pattern into input information according to the predetermined rule,and thereby, the plural instructions assigned to the patterns may beexecuted and the convenience may be improved.

Application Example 14

This application example of the invention is directed to the deviceaccording to Application Example 13, wherein the control unit performs apredetermined operation in response to the input information.

In the device, the control unit performs the operations assigned inresponse to the input information converted according to the respectivepatterns, and thereby, the plural functions assigned to the patterns maybe executed and the convenience may be improved.

Application Example 15

This application example of the invention is directed to the deviceaccording to Application Example 13 or 14, wherein the detection unitdetects opened and closed states of the eyelids at predetermined times.

In the device, the detection unit detects and determines two kinds ofstates of the opened state and the closed state of the eyelids atpredetermined times, and thereby, operability with high accuracy andless false detection of the states of the eyelids may be realized.

Application Example 16

This application example of the invention is directed to the deviceaccording to Application Example 15, wherein the respective patternsinclude patterns defined by combinations of the opened and closed statesof the eyelids at the respective plural times.

In the device, the patterns of the opened and closed states of theeyelids are the patterns defined by the combinations of the opened andclosed states of the eyelids at the plural predetermined times, andthereby, the instructions in response to the plural patterns may beperformed and the convenience may be improved.

Application Example 17

This application example of the invention is directed to the deviceaccording to Application Example 15 or 16, wherein the detection unitdetects the states of the right eyelid and the states of the lefteyelid, and the respective patterns include patterns defined bycombinations of the states of the right eyelid and the states of theleft eyelid.

In the device, the detection unit detects the opened and closed statesof the right eyelid and the left eyelid, and the patterns of the openedand closed states of the eyelids are patterns defined by thecombinations of the opened and closed states of the right eyelid and theleft eyelid, and thereby, the operations in response to the pluralpatterns may be performed and the convenience may be improved.

Application Example 18

This application example of the invention is directed to the deviceaccording to any of Application Examples 13 to 17, wherein the controlunit converts the respective patterns except some patterns of therespective patterns into the input information according to thepredetermined rule, but does not convert the some patterns.

In the device, the control unit converts the instructions assigned tothe respective patterns except some patterns of the patterns of thestates of the eyelids into the input information according to thepredetermined rule, but does not convert the instructions assigned tothe some patterns. Accordingly, in the device, when the instructionsassigned to the respective patterns are performed, execution of theinstruction not intended by the user may be avoided.

Application Example 19

This application example of the invention is directed to the deviceaccording to Application Example 13, wherein the control unit inputsinput data in response to the input information.

In the device, the control unit inputs input data in response to theinput information in which the states of the eyelids detected by thedetection unit are converted, and thereby, the user can input the inputdata without using hands and the operability may be improved.

Application Example 20

This application example of the invention is directed to the deviceaccording to any of Application Examples 13 to 19, which furtherincludes an operation part that receives operations of the user, whereinthe control unit converts combinations of the respective patterns andthe operations into the input information according to the predeterminedrule.

In the device, the control unit performs conversion into the inputinformation according to the combinations with the operations receivedby the operation part in addition to the patterns of the states of theeyelids, and thereby, false detection of the states of eyelids may beprevented and the user may give instructions in more patterns.

Application Example 21

This application example of the invention is directed to the deviceaccording to any of Application Examples 13 to 20, wherein the detectionunit detects the number of opened and closed times of the eyelids in apredetermined time, and the respective patterns include patterns definedby the number of opened and closed times of the eyelids in thepredetermined time.

In the device, the detection unit detects the number of opened andclosed times of the eyelids in the predetermined time and the patternsof the eyelids are defined by the number of opened and closed times, andthereby, the user may give instructions using simple state changes ofthe eyelids and the convenience may be improved.

Application Example 22

This application example of the invention is directed to the deviceaccording to any of Application Examples 13 to 21, wherein the detectionunit interprets the case where the time in which the eyelids are closedis shorter than a predetermined value as the opened state of the eyelidsof the states of the eyelids.

In the device, the detection unit interprets the case where the time inwhich the eyelids are closed is shorter than the predetermined value asthe opened state of the eyelids of the states of the eyelids, andthereby, false detection of the states of the eyelids may be avoidedwithout detection of eye blinks.

Application Example 23

This application example of the invention is directed to the deviceaccording to any of Application Examples 13 to 22, which furtherincludes a sensing part that senses motions of a head of the user,wherein the control unit converts combinations of the respectivepatterns and the motions of the head into the input informationaccording to the predetermined rule.

The device further includes the sensing part that senses motions of thehead of the user, and the control unit performs conversion into inputinformation according to the combinations of the respective patterns ofthe states of the eyelids and the sensed motions of the head of theuser. Accordingly, in the device, the user can perform operation withoutusing hands, false detection of the states of the eyelids may beprevented, and instructions in more patterns may be performed by thecombinations with the motions of the head.

Application Example 24

This application example of the invention is directed to a head mounteddisplay including an image display unit having an image light generationpart that generates and outputs image light based on image data and alight guide part that guides the image light to eyes of a user andallowing the user to visually recognize a virtual image, and the deviceaccording to any of Application Examples 13 to 23 connected to the imagedisplay unit, and transmitting the image data to the image display unitand controlling image display by the image display unit.

Since the head mounted display includes the control unit and the imagedisplay unit, an image display with high portability may be realized.

Note that the aspects of the invention can be realized in various forms,for example, in the forms of a device and a control method of thedevice, a head mounted display and a control method of the head mounteddisplay, a detection indicator, a state detection controller, an eyelidsensing controller, an information processing system, a head-mounteddisplay system, a computer program for realization of functions of thesemethods, devices, or the systems, a recording medium in which thecomputer program is recorded, data signals including the computerprogram and embodied within carrier wave, or the like.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described with reference to the accompanyingdrawings, wherein like numbers reference like elements.

FIG. 1 is an explanatory diagram showing an outer configuration of ahead mounted display 100 in the first embodiment of the invention.

FIG. 2 is an explanatory diagram showing a detailed configuration of animage display unit 20.

FIG. 3 is an explanatory diagram functionally showing a configuration ofthe head mounted display 100.

FIG. 4 is an explanatory diagram showing an example of a virtual imageto be recognized by a user.

FIG. 5 is a flowchart showing a flow of control processing of the headmounted display 100 in the first embodiment.

FIG. 6 is a flowchart showing a flow of control processing of the headmounted display 100 in a second embodiment.

FIGS. 7A and 7B are explanatory diagrams showing relationships betweentime and opened and closed states of eye lids in the second embodiment.

FIG. 8 is an explanatory diagram showing examples of functions inresponse to respective patterns of the eye lids of the user of the headmounted display 100 in the second embodiment.

FIG. 9 is an explanatory diagram showing examples of functions inresponse to respective patterns of the eye lids of the user of the headmounted display 100 in a modified example.

FIGS. 10A and 10B are explanatory diagrams showing examples of inputdata in response to states of the eye lids of the user of the headmounted display 100 in a modified example.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Next, embodiments of the invention will be explained in the followingorder according to examples.

A. First Embodiment

A-1. Device Configuration

A-2. Control Processing

B. Second Embodiment

C. Modified Examples

A. First Embodiment A-1. Device Configuration

FIG. 1 is an explanatory diagram showing an outer configuration of ahead mounted display 100 in the first embodiment of the invention. Thehead mounted display 100 is a display mounted on a head and also calledan HMD. The head mounted display 100 of the embodiment is an opticallytransmissive head mounted display by which a user can visually recognizea virtual image and directly visually recognize an outside view.

The head mounted display 100 includes an image display unit 20 thatallows the user to visually recognize a virtual image when the displayis mounted on the head of the user, and a control unit (controller) 10that controls the image display unit 20.

The image display unit 20 is a mounted body mounted on the head of theuser and has an eyeglasses shape in the embodiment. The image displayunit 20 has ear-fit parts 21 that function as temples, and a rightoptical panel 26 and a left optical panel 28 located in front of theright and left eye of the user, respectively, when the user wears theimage display unit 20. Further, a right display drive unit 22 isprovided in the connection part of the ear-fit part 21 for right ear andthe right optical panel 26, and a left display drive unit 24 is providedin the connection part of the ear-fit part 21 for left ear and the leftoptical panel 28. In the following explanation, the right optical panel26 and the left optical panel 28 are collectively simply referred to as“optical panels 26, 28”.

The image display unit 20 further has a right earphone 32 for right earand a left earphone 34 for left ear. The right earphone 32 and the leftearphone 34 are worn on the right and left ears, respectively, when theuser wears the image display unit 20.

The image display unit 20 further has a connection unit 40 forconnecting the image display unit 20 to the control unit 10. Theconnection unit 40 includes a main cable 48 connected to the controlunit 10, a right cable 42 and a left cable 44 formed by branching themain cable 48 into two, and a coupling part 46 provided at the branchpoint. The right cable 42 is connected to the right display drive unit22 and the left cable 44 is connected to the left display drive unit 24.The image display unit 20 and the control unit 10 perform transmissionof various signals via the connection unit 40. Connectors (not shown)mating each other are provided respectively at the end opposite to thecoupling part 46 in the main cable 48 and the control unit 10, and thecontrol unit 10 and the image display unit 20 are connected ordisconnected by mating or demating of the connecter of the main cable 48and the connector of the control unit 10.

The control unit 10 is a device that controls image display by the imagedisplay unit 20. The control unit 10 has a power switch 18 that switchesON/OFF of power, a lighting part 12 that indicates the operation statesof the head mounted display 100 (for example, the ON/OFF states ofpower) by light emission, a touch pad 14 that detects a finger operationof the user and outputs a signal in response to the finger operation,and a cross key 16 that detects pressing operation of the keycorresponding to the vertical and horizontal directions and outputs asignal in response to the operation. As the lighting part 12, forexample, one or plural LED lamps may be used. As the touch pad 14,various touch pads of electrostatic type, pressure detection type, oroptical type may be employed.

FIG. 2 is an explanatory diagram showing a detailed configuration of theimage display unit 20. The image display unit 20 has an infrared sensor50 that detects states of eyelids of the user. The infrared sensor 50has an infrared light emitting part 51 that outputs infrared lightstoward eyes of the user and an infrared light receiving part 56 thatreceives infrared lights reflected by the eyes of the user. The infraredlight emitting part 51 has a right infrared light emitting part 52 thatoutputs infrared light toward the right eye of the user and a leftinfrared light emitting part 54 that outputs infrared light toward theleft eye of the user. The right infrared light emitting part 52 isprovided in the right display drive unit 22 and the left infrared lightemitting part 54 is provided in the left display drive unit 24. Theinfrared light receiving part 56 is provided in the boundary part in themiddle of the right optical panel 26 and the left optical panel 28.

The infrared light output from the right infrared light emitting part 52is reflected by a right light guide plate 261, which will be describedlater, included in the image display unit 20, and enters the right eye62 of the user. The infrared reflected light from the right eye 62 ofthe incident light reaches the infrared light receiving part 56 and isdetected by the infrared sensor 50. The infrared light output from theleft infrared light emitting part 54 is the same as the infrared lightoutput from the right infrared light emitting part 52, and theexplanation is omitted here. Since the reflectance of the infrared lightis different between the state in which the user's eyes are opened andthe state in which the user's eyes are closed, and output values aredifferent in the infrared light receiving part 56. Accordingly, theinfrared sensor 50 may determine the state in which the user's eyes areopened and the state in which the user's eyes are closed.

FIG. 3 is an explanatory diagram functionally showing a configuration ofthe head mounted display 100. As shown in FIG. 3, the control unit 10has a CPU 140, a storage part 120 including a ROM, RAM, or the like, apower supply 130 that supplies power to the respective units of the headmounted display 100, an operation part 110 to be operated by the user(in this embodiment, the touch pad 14, the cross key 16, the powerswitch 18), and an interface 180 for connection to various externalequipment OA (for example, a personal computer, a cellular phoneterminal, and a game terminal) as supply sources of contents data suchas images (still images, moving images) and sound. As the power supply130, for example, a secondary cell may be used, and, as the interface180, for example, a USB interface, an interface for memory card, awireless LAN interface, or the like may be employed.

In the storage part 120, various computer programs are stored, and theCPU 140 reads out the computer programs from the storage part 120 andexecutes them, and thereby, functions as an operating system (OS) 150,an image processing part 160, a display control part 190, a detectioncontrol part 183, and a sound processing part 170.

The image processing part 160 generates a vertical synchronizing signalVSync, a horizontal synchronizing signal HSync, and image data Databased on the contents data input via the interface 180, and supply thesignals to the image display unit 20 via the connection unit 40.Specifically, the image processing part 160 generates the image dataData by acquiring image signals (for example, analog signals including30 frame images per second) contained in the contents data, separatingthe synchronizing signals of the vertical synchronizing signal VSync,the horizontal synchronizing signal HSync, etc. from the acquired imagesignals, and converting the analog image signals from which thesynchronizing signals have been separated into digital image signals.Note that the image processing part 160 may execute resolutionconversion processing, color compensation processing, etc. on the imagedata according to need.

The detection control part 183 is connected to the infrared sensor 50,and detects the output values of the infrared light received by theinfrared light receiving part 56. Further, the detection control part183 controls infrared light emission of the infrared light emitting part51. The detection control part 183 determines the state in which theuser's eyes are opened and the state in which the user's eyes are closedbased on the output values in the infrared light receiving part 56.According to the determination, the detection control part 183 generatesa control signal and supplies the control signal to the display controlpart 190. Note that the detection control part 183 and the infraredsensor correspond to a detection unit (detection part 185) according tothe invention.

The display control part 190 receives the control signal from thedetection part 185 or the like, generates control signals forcontrolling the right display drive unit 22 and the left display driveunit 24, supplies the signals to the image display unit 20 via theconnection unit 40, and thereby, controls an image display condition inthe image display unit 20. Specifically, the display control part 190individually controls drive ON/OFF of a right LCD 241 by a right LCDcontrol part 211 and drive ON/OFF of a right light source 221 by a rightlight source control part 201 and drive ON/OFF of a left LCD 242 by aleft LCD control part 212 and drive ON/OFF of a left light source 222 bya left light source control part 202 using the generated controlsignals, and thereby, controls generation or non-generation of therespective image lights by the right display drive unit 22 and the leftdisplay drive unit 24. For example, the display control part 190 allowsboth the right display drive unit 22 and the left display drive unit 24to generate image lights, only one of them to generate an image light,or both of them to generate no image light.

The sound processing part 170 acquires sound signals contained in thecontents data, amplifies the acquired sound signals, and supplies thesignals to the right earphone 32 and th left earphone 34 of the imagedisplay unit 20 via the connection unit 40.

The right display drive unit 22 of the image display unit 20 has theright light source 221 including an LED lamp, for example, the rightlight source control part 201 that drives the right light source 221based on the control signal supplied from the control unit 10, the rightliquid crystal display (LCD) 241 that converts the light output from theright light source 221 into image light representing an image, and theright LCD control part 211 that drive-controls the right LCD 241 basedon the vertical synchronizing signal VSync, the horizontal synchronizingsignal HSync, and the image data Data supplied from the control unit 10.The right LCD 241 includes a transmissive liquid crystal panel, forexample. The right LCD 241 and the right LCD control part 211 form aright-eye image generation part 244. The right display drive unit 22 hasa right projection system 251 that projects the generated image light.The right projection system 251 includes a collimater lens, for example.

The right optical panel 26 (FIG. 1) of the image display unit 20 has theright light guide plate 261, and the right light guide plate 261 guidesthe image light output from the right projection system 251 to the righteye 62 (FIG. 2) of the user while reflecting it along a predeterminedoptical path. Note that the right projection system 251 and the rightlight guide plate 261 correspond to a light guide part according to theinvention.

The left display drive unit 24 includes the left light source 222, theleft light source control part 202, the left LCD 242, the left LCDcontrol part 212, and a left projection system 252 like the rightdisplay drive unit 22. The configurations and functions of therespective elements included in the left display drive unit 24 are thesame as those of the respective elements included in the right displaydrive unit 22, and their explanation is omitted here. Further, the leftoptical panel 28 of the image display unit 20 has a left light guideplate 262, and the left light guide plate 262 guides the image lightoutput from the left projection system 252 to the left eye 64 (FIG. 2)of the user while reflecting it along a predetermined optical path. Theleft LCD 242 and the left LCD control part 212 form a left-eye imagegeneration part 246. The left projection system 252 and the left lightguide plate 262 correspond to the light guide part according to theinvention. As below, the right light source control part 201 and theleft light source control part 202 are also collectively simply referredto as “light source control parts 201, 202”, the right light source 221and the left light source 222 are also collectively simply referred toas “light sources 221, 222”, and the right-eye image generation part 244and the left-eye image generation part 246 are also collectively simplyreferred to as “image generation parts 244, 246”. Note that the lightsources 221, 222 and the image generation parts 244, 246 correspond toan image light generation part according to the invention.

In this manner, the image lights guided to the eyes of the user of thehead mounted display 100 are imaged on the retinas, and thereby, theuser visually recognize the virtual image. FIG. 4 is an explanatorydiagram showing an example of a virtual image to be recognized by auser. As shown in FIG. 4, the virtual image VI is visually recognizedwithin a field of view VR of the user of the head mounted display 100.Further, of the field of view VR of the user, with respect to the partsexcept the part in which the virtual image VI is visually recognized,the user can see the outside view SC through the right optical panel 26and the left optical panel 28. Note that, in the head mounted display100 of the embodiment, the outside view SC may be seen through on thebackground of the virtual image VI with respect to the part in which thevirtual image VI is visually recognized of the field of view VR of theuser.

Note that the right optical panel 26 and the left optical panel 28 maybe provided at the front side of the right light guide plate 261 and theleft light guide plate 262 (at the opposite side to the eyes of theuser), and may have dimming plates that can adjust light transmittance.If the dimming plates are provided, ease of visual recognition of thevirtual image may be adjusted by adjusting the light transmittance ofthe dimming plates and adjusting the amount of external light enteringthe eyes of the user.

A-2. Control Processing

FIG. 5 is a flowchart showing a flow of control processing of the headmounted display 100 in the first embodiment. When the head mounteddisplay 100 of the embodiment is used, contents data of moving pictures,images, etc. is supplied from the external equipment OA connected viathe interface 180 to the CPU 140. The display control part 190 generatescontrol signals based on the supplied contents data. The control signalsare supplied to the light source control parts 201, 202 and the imagegeneration parts 244, 246. The light source control parts 201, 202increase or decrease the luminance of the light sources 221, 222 basedon the supplied control signals. The image generation parts 244, 246convert the lights output from the light sources 221, 222 into imagelights based on the supplied control signals. The left projection system252 and the left light guide plate 262 guide the image lights to theeyes of the user. The image lights are visually recognized as a virtualimage by the user, and the sound processing part 170 included in the CPU140 reproduces the sound of the contents supplied from the externalequipment OA (step S310). Note that, in the embodiment, the head mounteddisplay 100 is the transmissive head mounted display, and thus, the usercan see through the outside view SC on the background of the virtualimage VI.

When the head mounted display 100 operates, the detection part 185detects the states of the eyelids (the opened and closed states of theeyelids) of the user (step S320). While the user views the contentssupplied from the external equipment OA, the detection part 185 maydetect the eyelids of the user kept closed. At this time, it isconsidered possible that the user is asleep. Accordingly, in theembodiment, the control unit 10 and the display control part 190 switchthe operation mode in response to the opened and closed states of theeyelids of the user detected by the detection part 185.

While the detection part 185 does not detect the eyelids of the userkept closed for 10 seconds or more (step S330: NO), the part determinesthat the user is not asleep and the normal operation of the head mounteddisplay 100 is continued (step S331).

At the normal operation of the head mounted display 100, if thedetection part 185 detects the eyelids of the user kept closed for 10seconds or more (step S330: YES), the part determines that the user isasleep. At this time, the display control part 190 reduces the luminanceof the light sources 221, 222 (step S340). By reducing the lights outputfrom the light sources 221, 222, the head mounted display 100 transitsto the operation mode with lower power consumption. The transition iscalled power-saving transition.

In the embodiment, the display control part 190 separately controls theright light source control part 201 and the left light source controlpart 202. The right light source control part 201 controls the rightlight source 221 and the left light source control part 202 controls theleft light source 222. Accordingly, the display control part 190 mayreduce the luminance of only one of the right light source 221 and theleft light source 222. Further, the display control part 190 mayseparately reduce the luminance of the right light source 221 and theleft light source 222.

If the detection part 185 detects the eyelids of the user opened within30 seconds after the luminance of the light sources 221, 222 is reduced(step S350: NO), the part determines the user has awaken from his or hersleep. At this time, the display control part 190 maximizes theluminance of the light sources 221, 222 (step S351). By maximizing theluminance of the light sources 221, 222, the head mounted display 100transits to the operation mode with higher power consumption. Thetransition is called recovery transition. Then, the display control part190 transits to the normal operation (step S331). Note that it is notnecessary that the increased amount of luminance of the light sources221, 222 is equal to the reduced amount of luminance of the lightsources 221, 222 in the processing at step S340. Further, the displaycontrol part 190 may gradually increase the luminance of the lightsources 221, 222 with the passage of time.

If the detection part 185 detects the eyelids of the user continuouslykept closed for 30 seconds or more after the luminance of the lightsources 221, 222 has been reduced (step S350: YES), the part determinesthat the user remains asleep. At this time, the display control part 190turns off the light sources 221, 222 (step S360). Thereby, the headmounted display 100 performs power-saving transition to the operationmode with much lower power consumption.

If the detection part 185 detects the eyelids of the user opened withinone minute after the light sources 221, 222 have been turned off (stepS370: NO), the part determines the user has awaken from his or hersleep. At this time, the display control part 190 allows the imagedisplay unit 20 to display a recuperation image (for example, an imagewith lower brightness) (step S371). Then, the display control part 190maximizes the luminance of the light sources 221, 222 (step S351) andtransits to the normal operation (step S331). Thereby, the head mounteddisplay 100 performs recovery transition to the operation mode withhigher power consumption. Note that, in the embodiment, if the luminanceof the light sources 221, 222 increases in a short time from theoff-state, the display control part 190 allows the image display unit 20to display the recuperation image because the eyes of the user aresignificantly strained. Note that the display control part 190 mayincrease the luminance of the light sources 221, 222 in a short timewithout displaying the recuperation image, or gradually increase theluminance of the light sources 221, 222 with the passage of time.

When the light sources 221, 222 are off, if the detection part 185detects the eyelids of the user continuously kept closed for one minuteor more (step S370: YES), the part determines that the user remainsasleep. At this time, the display control part 190 stops the operationsof the image generation parts 244, 246 (step S380). Thereby, the headmounted display 100 performs power-saving transition to the operationmode with still much lower power consumption.

In the embodiment, the display control part 190 separately controls theright-eye image generation part 244 and the left-eye image generationpart 246. Accordingly, the display control part 190 can stop theoperation of only one of the right-eye image generation part 244 and theleft-eye image generation part 246.

If the detection part 185 detects the eyelids of the user opened withinfive minutes after the operations of the image generation parts 244, 246have been stopped (step S390: NO), the part determines the user hasawaken from his or her sleep. At this time, the display control part 190starts the operations of the image generation parts 244, 246 again (stepS391). Then, the display control part 190 allows the image display unit20 to display the recuperation image (step S371), maximizes theluminance of the light sources 221, 222 (step S351), and transits to thenormal operation (step S331). Thereby, the head mounted display 100performs recovery transition to the operation mode with the higher powerconsumption.

If the detection part 185 detects the eyelids of the user continuouslykept closed for five minutes or more after the operations of the imagegeneration parts 244, 246 have been stopped (step S390: YES), the partdetermines that the user remains asleep. At this time, the control unit10 stops the operations of the image processing part 160, the displaycontrol part 190, the detection control part 183, and the soundprocessing part 170 (step S400). Thereby, the head mounted display 100performs power-saving transition to the operation mode with even stillmuch lower power consumption. Note that the operation mode in which theoperations of the image processing part 160, the display control part190, the detection control part 183, and the sound processing part 170are stopped is called “standby mode”. Further, the image processing part160, the display control part 190, the detection control part 183, andthe sound processing part 170 correspond to the respective ones offunctional parts that realize the plural functions of the control unit10 according to the invention.

In the standby mode, unless the power switch 18 of the operation unit110 is pressed down, the detection part 185 determines the user remainsasleep and the standby mode is continued (step S410: NO).

In the standby mode, if the power switch 18 of the operation unit 110 ispressed down (step S410: YES), the control unit 10 determines the userhas awaken from his or her sleep. At this time, the control unit 10 endsthe standby mode and the operations of the image processing part 160,the display control part 190, the detection control part 183, and thesound processing part 170 are started again (step S411). Then, thedisplay control part 190 allows the image display unit 20 to display therecuperation image (step S371), maximizes the luminance of the lightsources 221, 222 (step S351), and transits to the normal operation (stepS331). Thereby, the head mounted display 100 performs recoverytransition to the operation mode with the higher power consumption.

Further, in the embodiment, the storage part 120 has stored theinformation of the images of the contents data that the display controlpart 190 had allowed the image display unit to display before transitionto the standby mode.

Accordingly, when the control unit 10 ends the standby mode, theoperations of the image processing part 160, the display control part190, the detection control part 183, and the sound processing part 170are started again, and the display control part 190 may restart fromwhere interrupted based on the information of the images of the contentsdata stored in the storage part 120.

As described above, in the head mounted display 100 according to theinvention, the detection part 185 detects the opened and closed statesof the eyelids of the user. The display control part 190 increases ordecreases the luminance of the light sources 221, 222 and controls theoperations of the image generation parts 244, 246 in response to theopened and closed states of the eye lids. Further, the control unit 10transits to the standby mode or ends the standby mode in response to theopened and closed states of the eye lids. The display control part 190determines the user is asleep if the detection part 185 detects theeyelids of the user kept closed in a predetermined time. At this time,the display control part 190 and the control unit 10 perform thepower-saving transition. In the embodiment, as the power-savingtransition, the display control part 190 reduces the luminance of thelight sources 221, 222 and stops the operations of the image generationparts 244, 246. Further, the control unit 10 transits to the standbymode. On the other hand, in the operation mode with the lower powerconsumption than that of the normal operation, if the detection part 185does not detect the eyelids of the user kept closed in the predeterminedtime, the part determines the user has awaken from his or her sleep. Atthis time, the display control part 190 and the image display unit 20perform the recovery transition. In the embodiment, as the recoverytransition, the display control part 190 increases the luminance of thelight sources 221, 222 and starts the operations of the image generationparts 244, 246 again. The control unit 10 ends the standby mode.Therefore, in the head mounted display 100 of the embodiment, thedisplay control part 190 and the control unit 10 switch to theappropriate modes with different power consumption in response to thedetected opened and closed states of the eyelids of the user, which maysuppress the power consumption.

Further, in the head mounted display 100 of the embodiment, at thenormal operation, if the detection part 185 detects the eyelids of theuser kept closed for 10 seconds or more, the part determines that theuser is asleep. At this time, the display control part 190 reduces theluminance of the light sources 221, 222. Furthermore, when the luminanceof the light sources 221, 222 is reduced, if the detection part 185detects the eyelids of the user continuously kept closed for 30 secondsor more, the part determines that the user remains asleep. At this time,the display control part 190 turns off the light sources 221, 222.Therefore, the head mounted display 100 reduces the luminance of thelight sources 221, 222 and transits to the operation mode with the lowerpower consumption when the user is asleep, which may reduce the powerconsumption.

In addition, in the head mounted display 100 of the embodiment, when thelight sources 221, 222 are off, if the detection part 185 detects theeyelids of the user continuously kept closed for one minute or more, thepart determines that the user remains asleep. At this time, the displaycontrol part 190 stops the operations of the image generation parts 244,246. Therefore, the head mounted display 100 stops the operations of theimage generation parts 244, 246 and transits to the operation mode withstill much lower power consumption when the user remains asleep, whichmay reduce the power consumption.

Further, in the head mounted display 100 of the embodiment, when theoperations of the image generation parts 244, 246 are stopped, if thedetection part 185 detects the eyelids of the user continuously keptclosed for five minutes or more, the part determines that the userremains asleep. At this time, the control unit 10 transits to thestandby mode. Therefore, the head mounted display 100 stops theoperations of the image processing part 160, the display control part190, the detection control part 183, and the sound processing part 170and transits to the operation mode with even still much lower powerconsumption when the user remains asleep, which may reduce the powerconsumption.

Furthermore, as described above, the head mounted display 100 of theembodiment performs the power-saving transition in a stepwise manner inresponse to the time in which the eyelids of the user are kept closed.It may be considered that the user is more likely to wake up when theelapsed time after the eyelids of the user are closed is shorter thanwhen the elapsed time is longer. It is desirable that, when the userwakes up, the time taken for recovery transition to the normal operationis shorter. Therefore, the head mounted display 100 transits to theoperation modes with lower power consumption in a stepwise manner inresponse to the elapsed time after the eyelids of the user are closed,which may promptly recover to the normal operation while reducing thepower consumption.

In addition, in the head mounted display 100 of the embodiment, when theluminance of the light sources 221, 222 is reduced, if the detectionpart 185 does not detect the eyelids of the user kept closed for further30 seconds or more, the part determines the user has awaken from his orher sleep. At this time, the display control part 190 maximizes theluminance of the light sources 221, 222. Further, when the light sources221, 222 are off, if the detection part 185 does not detect the eyelidsof the user kept closed for further one minute or more, the partdetermines the user has awaken from his or her sleep. At this time, thedisplay control part 190 allows the image display unit 20 to display therecuperation image. Then, the display control part 190 maximizes theluminance of the light sources 221, 222. Therefore, the head mounteddisplay 100 may reduce the power consumption when the user is asleep andmay promptly recover to the normal operation when the user awakes fromhis or her sleep.

Further, in the head mounted display 100 of the embodiment, when theoperations of the image generation parts 244, 246 are stopped, if thedetection part 185 does not detect the eyelids of the user opened forfurther five minutes or more, the part determines the user has awakenfrom his or her sleep. At this time, the display control part 190 startsthe operations of the image generation parts 244, 246 again. Then, thedisplay control part 190 allows the image display unit 20 to display therecuperation image and maximizes the luminance of the light sources 221,222. Therefore, the head mounted display 100 may reduce the powerconsumption when the user is asleep and may promptly recover to thenormal operation when the user awakes from his or her sleep.

Furthermore, in the head mounted display 100 of the embodiment, in thestandby mode, if the power switch 18 of the operation unit 110 ispressed down, the control unit 10 determines the user has awaken fromhis or her sleep. At this time, the control unit 10 ends the standbymode and the operations of the image processing part 160, the displaycontrol part 190, the detection control part 183, and the soundprocessing part 170 are started again. Then, the display control part190 allows the image display unit 20 to display the recuperation imageand maximizes the luminance of the light sources 221, 222. Therefore,the head mounted display 100 may reduce the power consumption when theuser is asleep and may promptly recover to the normal operation when theuser awakes from his or her sleep.

In addition, the head mounted display 100 performs recovery transitionin a stepwise manner in response to the time in which the eyelids of theuser are kept closed as described above. It may be considered that thesleep of the user is deeper when the elapsed time after the eyelids ofthe user are closed is longer. Accordingly, when the detection part 185detects the eyelids of the user opened from the closed state, if theluminance of the light sources 221, 222 increases in a short time, theeyes of the user are significantly strained. Therefore, the head mounteddisplay 100 performs the recovery transition in the stepwise manner, andthereby, the eyes of the user are less strained.

Further, in the head mounted display 100 of the embodiment, the imageprocessing part 160 stores the information of the contents that thedisplay control part 190 have allowed the image display unit 20 todisplay in the storage part 120 before transition to the standby mode.The information of the contents includes frame numbers or frame data,for example. Accordingly, when the control unit 10 ends the standbymode, the operations of the image processing part 160, the displaycontrol part 190, the detection control part 183, and the soundprocessing part 170 are started again, and the display control part 190may restart from where interrupted based on the information of theimages of the contents stored in the storage part 120. Therefore, in thehead mounted display 100, the user may efficiently restart the contentsintended to view.

Furthermore, in the head mounted display 100 of the embodiment, thedisplay control part 190 separately controls the right light sourcecontrol parts 201 and the left light source control part 202. The rightlight source control part 201 controls the right light source 221 andthe left light source control part 202 controls the left light source222. Accordingly, the display control part 190 may reduce the luminanceof only one of the right light source 221 and the left light source 222.Further, the display control part 190 separately controls the right-eyeimage generation part 244 and the left-eye image generation part 246.Accordingly, the display control part 190 can stop the operation of onlyone of the right-eye image generation part 244 and the left-eye imagegeneration part 246. Therefore, the head mounted display 100 separatelycontrols the right light source 221 and the left light source 222 andthe right-eye image generation part 244 and the left-eye imagegeneration part 246, which may realize flexible reduction of powerconsumption.

B. Second Embodiment

FIG. 6 is a flowchart showing a flow of control processing of the headmounted display 100 in the second embodiment. The head mounted display100 in the second embodiment is adapted to be operated by the user usingeyelids without using hands. The control processing is processing by thehead mounted display 100 of performing control in response to theoperation using the eyelids of the user.

In the control processing of the head mounted display 100, the detectionpart 185 detects opened and closed states of the right eyelid 72 (FIG.2) of the right eye 62 and the left eyelid 74 of the left eye 64 (stepS510). FIGS. 7A and 7B are explanatory diagrams showing relationshipsbetween time and opened and closed states of eye lids in the secondembodiment. FIG. 7A shows an example of a detected open/close signal ES1representing the opened and closed states of the right eyelid 72 (or theleft eyelid 74) in the respective times detected by the infrared sensor50. FIG. 75 shows an example of an interpreted open/close signal ES2representing the opened and closed states of the right eyelid 72 (or theleft eyelid 74) in the respective times interpreted by the detectioncontrol part 183 that has received the detection result by the infraredsensor 50. FIG. 7B also shows the detected open/close signal ES1 bydashed-dotted lines for reference. In FIGS. 7A and 7B, the state inwhich the eyelid is closed is indicated by “0” and the state in whichthe eyelid is opened is indicated by “1”. Note that, as shown in FIG. 3,the infrared sensor 50 and the detection control part 183, i.e., thedetection part 185 is attached to the image display unit 20, however, ispart of the configuration of the control unit 10.

Usually, the user of the head mounted display 100 opens both eyelids.The infrared sensor 50 constantly detects the opened and closed statesof the eyelids of the user and output signals representing the openedand closed states of the eyelids (the detected open/close signals ES1shown in FIG. 7A) to the detection control part 183. The detectioncontrol part 183 determines the opened and closed states of the eyelidsof the user based on the detected open/close signals ES1, and, forexample, if the state in which only one of the right eyelid 72 and theleft eyelid 74 of the user is closed is detected, with the detectiontime as the start time T0, detects the opened and closed states of therespective eyelids at the first detection time T1 0.5 seconds after thestart time T0 and at the second detection time T2 further one secondafter. Note that, although the time from the start time T0 to the firstdetection time T1 has been set to 0.5 seconds and the time from thefirst detection time T1 to the second detection time T2 has been set toone second, the times are not limited to the values, but the times maybe set freely by the user.

FIGS. 7A and 7B show the examples in which the user of the head mounteddisplay 100 of the second embodiment performs operation of opening theeyelid at the first detection time T1 and closing the eyelid at thesecond detection time T2.

Usually, the user of the head mounted display 100 may unconsciouslyblink even when performing operation of opening the eyelids. The sectionof the time BT shown in FIG. 7A is a section in which the eyelid isclosed by blinking. As shown in FIG. 7A, if the first detection time T1overlaps with the section of the blink time BT, the infrared sensor 50detects the state in which the eyelid is closed at the first detectiontime T1.

As shown in FIG. 7B, the detection control part 183 converts thedetected open/close signal ES1 detected by the infrared sensor 50 intothe interpreted open/close signal ES2. In order not to determine theeyelid closed even when detecting an eye blink, the detection controlpart 183 determines an eye blink if the time in which the eyelid isclosed is shorter than a preset threshold value Tth and determines theopened state, not the closed state of the eyelid. In the head mounteddisplay 100 in the second embodiment, the preset threshold value Tth isset to 0.3 seconds in consideration of the blink time BT (0.10-0.15seconds), and the detection control part 183 interprets the detectionresults to remove the eye blinks from the opened and closed states ofthe eyelids and the detection part 185 detects the opened and closedstates of the eyelids. Note that the preset threshold value Tth is notlimited to 0.3 seconds.

After the detection part 185 detects the opened and closed states of theeyelids, the display control part 190 determines whether or not therespective patterns combining the opened and closed states of the righteyelid 72 and the left eyelid 74 at the first detection time T1 and thesecond detection time T2 have assigned functions (step S520).

FIG. 8 is an explanatory diagram showing examples of functions inresponse to the respective patterns of the eye lids of the user of thehead mounted display 100 in the second embodiment. The detection part185 detects the opened and closed states of the right eyelid 72 and theleft eyelid 74 at the first detection time T1 and the second detectiontime T2, and thus, there are 16 patterns of the opened and closed statesof the eyelids from pattern P01 to pattern P16.

As shown in FIG. 8, there are patterns having assigned functions andpatterns having no assigned functions. For example, the pattern P16 isthe same as the normal state in which the eyelids of the eyes areconstantly opened, and thus, it is preferable not to assign any functionto the pattern P16. Further, the difference in opened and closed statesof eyelids between the pattern P03 and the pattern P04 is in that theleft eyelid 74 changes from the opened state to the closed state at thesecond detection time T2 or remains closed. In the case where thefunction assigned to the pattern P03 is executed, if the left eyelid 74is held opened to the second detection time T2, the display control part190 converts the function into a control signal for the functionassigned to the pattern P04 and supplies it to the image display unit20. Accordingly, in the head mounted display 100 of the secondembodiment, the function different from the function intended by theuser may be executed, and thus, it is preferable not to assign anyfunction to one of the pattern P03 and the pattern P04. Further,similarly, it is preferable not to assign any function to the patternP01, the pattern P04, the pattern P13 in which the eyelids do not changefrom the first detection time T1 to the second detection time T2.Furthermore, depending on the interval between the first detection timeT1 and the second detection time T2, in order to prevent the executionof the function not intended by the user, it is preferable not to assignany function to some patterns. Note that the functions assigned inresponse to the patterns are not limited to the functions shown in FIG.8.

If no function according to the pattern detected by the detection part185 is assigned (step S520: NO), the control processing of the headmounted display 100 in the second embodiment is ended.

If a function is assigned in response to the pattern detected by thedetection part 185 (step S520: YES), the display control part 190executes the function assigned to the pattern (step S530). For example,if the user opens the right eyelid 72 and closes the left eyelid 74 atthe first detection time. T1, and then, closes the right eyelid 72 andopens the left eyelid 74 at the second detection time T2, the displaycontrol part 190 converts the function assigned to the pattern P10 intoa control signal and performs the operation of “Back” of Browser. Notethat the control signal in the second embodiment corresponds to inputinformation according to the invention.

As shown in FIG. 8, in addition to the pattern P10, the functions areassigned in response to the respective patterns of the pattern 202, thepattern P03, the pattern P05, the pattern P07, the pattern P12, and thepattern P15. The function assigned to the pattern P12 is the functionperformed for the program such as a browser like that to the pattern210. The other functions of the respective patterns are the samefunctions as the operation modes in the head mounted display 100 of thefirst embodiment, and executed regardless of the contents of the imagesbeing visually recognized by the user (for example, the Internet, movingimage contents, etc.) unlike the functions of the pattern P10 and thepattern P12.

As explained above, in the head mounted display 100 in the secondembodiment, the detection part 185 detects the states of the eyelids ofthe user and the display control part 190 executes the functionsassigned to the respective detected patterns of the eyelids, andthereby, the user can perform operations without using hands and theconvenience may be improved.

Further, in the head mounted display 100 in the second embodiment, thedetection control part 183 detects the opened and closed states of theright eyelid 72 and the left eyelid 74 at the first detection time T1and the second detection time T2, and the detection part 185 detects thepattern of the opened and closed states of the eyelids. The displaycontrol part 190 converts the function assigned to the pattern of theeyelids into the control signal and performs an operation. Accordingly,in the head mounted display 100 in the second embodiment, the user canperform operations without using hands and can execute plural functionsassigned in response to the patterns of the opened and closed states ofthe eyelids, and thereby, the convenience may be more improved. Further,in the head mounted display 100 in the second embodiment, the detectionpart 185 detects only the two kinds of states of the opened state andthe closed state of the eyelids, and thereby, operability with highaccuracy and less false detection may be realized. Furthermore, the headmounted display 100 in the second embodiment includes the control unit10 and the image display unit 20, and thereby, an image display withhigh portability may be realized.

Further, in the head mounted display 100 in the second embodiment, thedetection control part 183 determines an eye blink if the time in whichthe eyelid is closed is shorter than the preset threshold value Tth andinterprets it as the opened state of the eyelid, not the closed state,and the detection part 185 detects the opened and closed states of theeyelid. Accordingly, in the head mounted display 100 in the secondembodiment, the detection part 185 does not detect the closed state ofthe eyelid of the eye blink, and thus, false detection of the state ofthe eyelid may be avoided.

Furthermore, in the head mounted display 100 in the second embodiment,the pattern P16 is the same as the normal state in which the eyelids ofthe eyes are constantly opened, and thus, no function is assigned to thepattern P16, and no function is assigned to one of the pattern P03 andthe pattern P04. In addition, no function is assigned to the patternP01, the pattern P04, the pattern P13 in which the eyelids do not changefrom the first detection time T1 to the second detection time T2.Accordingly, in the head mounted display 100 in the second embodiment,since no function is assigned to some patterns of the patterns of theeyelids, when the functions assigned to the respective patterns areexecuted, execution of the function not intended by the user may beavoided.

C. Modified Examples

Note that the invention is not limited to the above describedembodiments, but can be implemented in various forms without departingfrom the scope of the invention. For example, the following changes canbe made.

C1. Modified Example 1

The configurations of the head mounted display 100 in the embodimentsare just examples and various changes can be made. In the firstembodiment, when the detection part 185 detects the eyelids of the userkept closed in the predetermined time, the part determines the user isasleep. However, the user may just close the eyelids, not sleep or mayhalf close the eyes. Accordingly, the display control part 190 may allowthe image display unit 20 to display a message of power-savingtransition immediately before the power-saving transition. Further, atthe same time, the sound processing part 170 gives a message ofpower-saving transition in sound. Note that the message of power-savingtransition may be given only by display of the display control part 190or sound of the sound processing part 170. If the detection part 185detects the eyelids of the user opened after notification of thepower-saving transition to performance of the power-saving transition,the display control part 190 does not perform the power-savingtransition. Therefore, the head mounted display 100 may not performpower-saving transition only depending on the states of the eyelids ofthe user detected by the detection part 185, but may performpower-saving transition after confirmation of the will of the user.

C2: Modified Example 2

In the first embodiment, power-saving transition and recovery transitionare stepwise transitions in response to the states of the eyelids of theuser, however, power-saving transition and recovery transition not inthe stepwise manner may be employed. For example, the head mounteddisplay 100 may suddenly turn off the power switch 18, suddenly transitto the standby mode, or suddenly stop the image generation parts 244,246 in response to the states of the eyelids of the user.

C3: Modified Example 3

In the embodiments, the detection part 185 detects the states of theeyelids of the user using the infrared sensor 50, however, may detectusing other detection devices. For example, the detection part 185 maydetect the states of the eyelids using an imaging device such as a CCD.Further, the detection part 185 may detect the states of the eyelids bydetecting the motion of the muscles at an eye blink using pressuresensors provided in the ear-fit parts 21.

In the embodiments, the detection part 185 realizes detection of thestates of the eyelids of the user using the infrared sensor 50, however,the detection can be made by known various methods. For example, thedetection may be realized by a detection method using a reflectionsensor of placing an infrared light emitting diode and a photodiode infront of the eyes (http://www.ee.toyota-ct.ac.jp/˜ozeki/index.html).

C4: Modified Example 4

The predetermined time in which the eyelids of the user are kept closed,which is the criterion for power-saving transition in the firstembodiment, is just an example, and may be set to an arbitrary value.Further, when recovery transition is determined, recovery transition maybe performed if the eyelids of the user are opened in a predeterminedtime (for example, three seconds).

C5: Modified Example 5

In the first embodiment, the control unit 10 ends the standby mode ifthe power switch 18 of the operation part 110 is pressed down, however,the standby mode may be ended by other operation. The control unit 10may end the standby mode if other button than the power switch 18 of theoperation part 110 is pressed down. Further, an acceleration sensor maybe provided in the operation part 110, and the control unit 10 may endthe standby mode if the operation part 110 detects predeterminedacceleration.

C6: Modified Example 6

In the embodiments, the control unit 10 stops the operations of theimage processing part 160, the display control part 190, the detectioncontrol part 183, and the sound processing part 170 in the standby mode,however, not all of the operations of the functional parts may bestopped. For example, the control unit 10 may reduce the powerconsumption by stopping only the operation of the image processing part160.

Further, in the embodiments, the operating system 150, the imageprocessing part 160, the display control part 190, the detection part185, and the sound processing part 170 are realized by the CPU 140executing software, however, they may be realized by hardware such as anelectronic circuit. For example, when the image processing part 160 andthe display control part 190 are realized by an electronic circuit, thecontrol unit 10 may reduce the power consumption by stopping theoperations of the image processing part 160 and the display control part190.

C7: Modified Example 7

In the second embodiment, specific times are exemplified as the timeswhen the detection part 185 detects the states of the eyelids of theuser, however, the detection times are not limited to those. The lengthof the time is not limited, and the detection times may be set by theuser of the head mounted display 100. The states of the eyelids detectedat the start time T0 are not limited to the states of the embodiment,and the states of the eyelids different from those of the secondembodiment may be detected by the detection part 185 for setting of thestart time T0. For example, the start time T0 may be set to when thecondition that the eyelids are closed in a predetermined period isdetected or when a predetermined number of eye blinks are detected in apredetermined period. Further, the start time T0 may be set to the timewhen the operation part 110 receives the operation, or the start time T0may be set to the time when an acceleration sensor provided in the headmounted display 100 or the like senses the motion of the head of theuser. The states of the eyelids detected by the detection part 185 arenot limited to the opened and closed states of the eyelids of theembodiment, however, the states of the eyelids may be detected byopening areas of the eyelids using a CCD or the like. Further, thefunctions assigned in response to the states of the eyelids are notlimited to those of the embodiment, however, for example, functions ofstopping and reproducing music contents may be employed.

In addition, the patterns of the opened and closed states of the righteyelid 72 and the left eyelid 74 are not limited to those of theembodiment. FIG. 9 is an explanatory diagram showing examples offunctions in response to respective patterns of the eye lids of the userof the head mounted display 100 in a modified example. In the patternsshown in FIG. 9, unlike the patterns shown in FIG. 8 in the secondembodiment, the display control part 190 assigns functions in responseto the patterns of only the opened and closed states of the right eyelid72 and the left eyelid 74 at the first detection time T1. Further, inthe patterns shown in FIG. 9, the assigned functions are differentdepending on the contents being viewed by the user even for the samepatterns. For example, the function assigned to the pattern P21 performsthe operation of “back” when the user views a browser, and performs theoperation of “go to the next chapter” when the user reproduces themoving images. Accordingly, in the head mounted display 100 in themodified example, the user can perform operations without using handsand the convenience may be improved. Note that the pattern P24 shown inFIG. 9 is the normal state in which eyelids are opened, and thus, nofunction is assigned to the pattern P24.

FIGS. 10A and 10B are explanatory diagrams showing examples of inputdata in response to states of the eye lids of the user of the headmounted display 100 in a modified example. FIG. 10A shows an example ofinput data in response to the patterns of the eyelids. FIG. 10B showsrelations between the opened and closed states of the eyelids and thepatterns. In FIGS. 10A and 10B, the display control part 190 convertsthe patterns of the opened and closed states of the eyelids into inputdata. As shown in FIG. 10A, the address to be input is designated by theopened or closed state of the right eyelid 72 and the input data to beinput is designated by the opened or closed state of the left eyelid 74.For example, in the head mounted display 100 in the modified example,the detection part 185 detects the opened and closed states of theeyelids of the right eyelid 72 and the left eyelid 74 at intervals of0.5 seconds from the start time T0 alternately at four times. As shownin FIG. 10B, there are 16 patterns of the respective eyelids in themodified example expressed as binary numbers. As shown in FIG. 10A, theaddress designated by the combinations of the opened and closed statesof the right eyelid 72 is “6” in hex and the input data by thecombinations of the opened and closed states of the left eyelid 74 is“F”. The display control part 190 inputs the input data corresponding to“F” to the address corresponding to “6”. Accordingly, in the headmounted display 100 in the modified example, the user can input theinput data without using hands, and the convenience may be improved.Note that the input data and the designated address correspond to theinput information according to the invention. Further, instructions inresponse to the states of the eyelids include designation of the addressand input of the input data.

Further, in the second embodiment, the display control part 190 executesthe function assigned in response to the detected pattern of the openedand closed states of the eyelids, however, may convert the function intoa control signal according to a combination with the operation receivedby the operation part 110 (the touch pad 14, the cross key 16, the powerswitch 18, etc.) provided in the control unit 10 in addition to thepattern of the eyelids. For example, if the detection part 185 is set todetect the states of the eyelids only when the cross key 16 of thecontrol unit 10 is pressed down, the user may freely determine the timesof the start time T0, the first detection time T1, and the seconddetection time T2, and thus, false detection of the states of theeyelids may be avoided. Further, in the combinations of the patterns ofeyelids and the operations, more functions may be assigned than thefunctions assigned only by the patterns of eyelids.

Furthermore, the display control part 190 may convert a function into acontrol signal according to a combination with a motion of the head ofthe user sensed by an acceleration sensor or the like connected to thecontrol unit 10 and provided in the head mounted display 100 in additionto the pattern of the eyelids. In the head mounted display 100 in themodified example, the user can perform operations without using handsand the user can freely determine the times of detection, and thus,false detection of the states of eyelids may be avoided and morefunctions may be assigned. Further, the function may be furtherconverted into a control signal by combining it with the operationreceived by the operation part 110. Note that the control signal of thehead mounted display 100 in the modified example corresponds to theinput information according to the invention.

In addition, in the second embodiment, the detection part 185 detectsthe opened and closed states of the respective eyelids at the firstdetection time T1 and the second detection time T2, however, thedetection condition of the eyelids is not limited to that. For example,in the head mounted display 100 of the second embodiment, the detectionpart 185 may detect the number of opened and closed times of the eyelidsin the section from the start time T0 to the second detection time T2and the display control part 190 may execute the function assigned tothe pattern in response to the number. In the head mounted display 100of the modified example, the display control part 190 executes thefunction assigned to the pattern according to the number of opened andclosed times of the eyelids in a predetermined time detected by thedetection part 185, and thereby, the user can perform operations usingsimple state changes of the eyelids and the convenience may be improved.

Further, in the second embodiment, the display control part 190 executesthe function assigned in response to the pattern of the opened andclosed states of the right eyelid 72 and the left eyelid 74, however,may execute the function in response to the opened or closed state ofonly the right eyelid 72 or only the left eyelid 74. Furthermore, in thesecond embodiment, the timing of detection of the states of the eyelidsby the detection part 185 is based on the relation with the time,however, not necessarily. For example, the display control part 190 mayinform the user of the times when the detection part 185 detects thestates of the eyelids by sound. Further, the display control part 190may inform the user of the times of detection of the states of theeyelids by displaying an image displayed on the image display unit 20,for example, a count to the detection of the states of the eyelids onthe image display unit 20 or otherwise, or allows the image display unit20 to display the detected pattern at the same time.

C8. Modified Example 8

In the embodiments, the cross key 16 provided in the control unit 10 maybe omitted, or another operation interface such as an operation stickmay be provided in addition to the cross key 16 and the touch pad 14.Further, the control unit 10 may have a configuration to which inputdevices such as a keyboard and a mouse can be connected and receiveinputs from the keyboard and the mouse. Furthermore, a communicationpart such as a wireless LAN may be provided in the control unit 10.

Further, in the embodiments, the control unit 10 and the image displayunit 20 are wired-connected by the connection part 40, however, thecontrol unit 10 and the image display unit 20 may be connected via awireless LAN, infrared communication, or a signal transmission path suchas Bluetooth.

In the embodiments, the image display unit 20 is the HMD for wearinglike eyeglasses, however, the image display unit 20 may be a typicalflat display (a liquid crystal display, a plasma display, an organicelectro-luminescence (EL) display, or the like). Also, in this case, theconnection between the control unit 10 and the image display unit 20 maybe connection via a wired signal transmission path or a wireless signaltransmission path. According to the configuration, the control unit 10may be used as a remote of the typical flat display.

Further, as the image display unit, in place of the image display unit20 worn like eyeglasses, an image display unit having another shape suchas an image display unit worn like a hat, for example, may be employed.Furthermore, the earphones 32, 34 may be omitted.

In addition, in the embodiments, as a configuration of generating imagelight, the LCD and the light source are used, however, in place of them,another display device such as an organic EL display may be employed.

Further, in the embodiments, the head mounted display 100 is theoptically transmissive head mounted display of a binocular type,however, the invention may be applied to other types of head mounteddisplays such as a video transmissive type, a non-transmissive type, ora monocular type. Furthermore, the invention may be applied not only tothe operation of the head mounted display but also to operation ofequipment such as a personal computer, a projector or a television.

Further, in the embodiments, the head mounted display 100 has theconfiguration including the control unit 10 and the image display unit20, however, not limited to the configuration. Even in other equipmentthan the head mounted display 100 (for example, a portable terminal, amachine operation unit in a factory or the like, a medical terminal, aninstrument such as an oscilloscope, a projection image display, amonitor of a personal computer, a television, or the like), the controlunit 10 gives instructions in response to the states of the eyelidsdetected by the detection part 185, and thereby, the user can performoperations without using hands and the convenience may be improved.

Further, in a detection indicator, a state detection controller, aneyelid sensing controller, or the like, the control unit givesinstructions in response to the detected states of the eyelids, andthereby, the user can perform operations without using hands and theconvenience may be improved. For example, in a remote of the equipment,locking of operation may be set to perform on/off-control in opening andclosing of the eyelids. Further, in a mouse pointer of a computer,double blink of both eyes within a predetermined time may be control ofperforming the same instruction as the double click of the mouse, orcontrol of the mouse pointer moving to the right at a predeterminedspeed when only the right eyelid is closed may be set.

Further, in the embodiments, part of the configuration realized byhardware may be replaced by software, or, conversely, part of theconfiguration realized by software may be replaced by hardware. Forexample, in the embodiments, the image processing part 160 and the soundprocessing part 170 are realized by the CPU 140 reading out andexecuting the computer program, however, these functional parts may berealized by a hardware circuit.

Furthermore, in the case where part or all of the functions of theinvention are realized by software, the software (computer programs) maybe provided in storage in a computer-readable recording medium. In thisinvention, “computer-readable recording medium” includes not only aportable recording medium such as a flexible disc or CD-ROM but also aninternal memory device within a computer such as various kinds of RAMsand ROMs and an external memory device fixed to the computer such as ahard disc.

C9. Modified Example 9

In the embodiments, the head mounted display 100 may guide image lightsrepresenting the same image to the right and left eyes of the user andallow the user to visually recognize a two-dimensional image, or mayguide image lights representing different images to the right and lefteyes of the user and allow the user to visually recognize athree-dimensional image.

Further, in the embodiments, the contents data is supplied from theexternal equipment OA, however, contents data may be stored in thestorage part 120 of the control unit 10 of the head mounted display 100and image display may be performed based on the contents data stored inthe storage part 120.

Furthermore, of the component elements in the above describedembodiments, the elements other than the elements described inindependent claims are additional elements, and may appropriately beomitted or combined.

The entire disclosure of Japanese Patent Application Nos. 2011-066383,filed Mar. 24, 2011 and 2011-255791, filed Nov. 24, 2011 are expresslyincorporated by reference herein.

What is claimed is:
 1. A device comprising: a detection unit thatdetects states of eyelids of a user; and a control unit that givesinstructions in response to the states of the eyelids of the userdetected by the detection unit, wherein the control unit performs astep-wise power saving process when it is determined, based on thestates of the eyelids of the user detected by the detection unit, thatthe user has entered a sleeping state, the step-wise process includingat least three different power saving states.
 2. A head mounted displaycomprising: an image display unit having an image light generation partthat generates and outputs image light based on image data and a lightguide part that guides the image light to eyes of the user and allowsthe user to visually recognize a virtual image; and the device accordingto claim 1 connected to the image display unit, wherein the control unittransmits the image data to the image display unit and controls imagedisplay by the image display unit.
 3. The head mounted display accordingto claim 2, wherein the control unit performs a power-saving transitionto a said power saving state with lower power consumption when thedetection unit detects the eyelids of the user kept closed for apredetermined time.
 4. The head mounted display according to claim 3,wherein the control unit performs a recovery transition to a said powersaving state with higher power consumption when the detection unitdetects the eyelids of the user opened a predetermined time after thepower-saving transition.
 5. The head mounted display according to claim3, wherein the image light generation part includes a light source thatoutputs light and an image generation part that converts the lightoutput from the light source into the image light representing an imagebased on the image data, and the control unit reduces the light outputfrom the light source at the power-saving transition.
 6. The headmounted display according to claim 3, wherein the control unit includesa functional part that realizes plural functions and stops a partialoperation of the functional part at the power-saving transition.
 7. Thehead mounted display according to claim 6, wherein the control unitfurther has an operation part and starts the stopped partial operationof the functional part again when the operation part receives a specificoperation after the power-saving transition.
 8. The head mounted displayaccording to claim 7, wherein the control unit stores information of theimage displayed by the image display unit at the power-saving transitionand restarts display from the stored image displayed at the power-savingtransition at a recovery transition to a said power saving state withhigher power consumption when the detection unit detects the eyelids ofthe user opened a predetermined time after the power-saving transition.9. The device according to claim 1, wherein the control unit convertsrespective patterns of the states of the eyelids detected by thedetection unit into input information according to a predetermined rule.10. The device according to claim 9, wherein the control unit performs apredetermined operation in response to the input information.
 11. Thedevice according to claim 9, wherein the detection unit detects openedand closed states of the eyelids at predetermined times.
 12. The deviceaccording to claim 11, wherein the respective patterns include patternsdefined by combinations of the opened and closed states of the eyelidsat the respective predetermined times.
 13. The device according to claim11, wherein the detection unit detects the states of the right eyelidand the states of the left eyelid, and the respective patterns includepatterns defined by combinations of the states of the right eyelid andthe states of the left eyelid.
 14. The device according to claim 9,wherein the control unit inputs input data in response to the inputinformation.
 15. The device according to claim 9, further comprising anoperation part that receives operations of the user, wherein the controlunit converts combinations of the respective patterns and the operationsinto the input information according to the predetermined rule.
 16. Thedevice according to claim 9, wherein the detection unit interprets as anopened state of the eyelids of the states of the eyelids the case wherethe time in which the eyelids are closed is shorter than a predeterminedvalue.
 17. The device according to claim 9, further comprising a sensingpart that senses motions of a head of the user, wherein the control unitconverts combinations of the respective patterns and the motions of thehead into the input information according to the predetermined rule. 18.A head mounted display comprising: an image display unit having an imagelight generation part that generates and outputs image light based onimage data and a light guide part that guides the image light to eyes ofthe user and allows the user to visually recognize a virtual image; andthe device according to claim 9 connected to the image display unit, andtransmitting the image data to the image display unit and controllingimage display by the image display unit.
 19. A control method of adevice, the method comprising: detecting states of eyelids of a user;giving instructions to the device in response to the detected states ofthe eyelids of the user; and performing a step-wise power saving processwhen it is determined, based on the detected states of the eyelids ofthe user, that the user has entered a sleeping state, the step-wiseprocess including at least three different power saving states.
 20. Acontrol method of a head mounted display including an image display unithaving an image light generation part that generates and outputs imagelight based on image data and a light guide part that guides the imagelight to eyes of a user and allows the user to visually recognize avirtual image, the control method comprising: performing control suchthat (i) the image data is transmitted to the image display unit and(ii) image display by the image display unit is controlled; detectingstates of eyelids of the user during a detection period; and performinga step-wise power saving process when it is determined, based on thestates of the eyelids of the use detected during the detection period,that the user has entered a sleeping state, the step-wise processincluding at least three different power saving states.
 21. The deviceaccording to claim 1, wherein it is determined that the user has enteredthe sleeping state when the eyelids of the user are determined to beclosed for a specified duration of time.
 22. The method according toclaim 19, wherein it is determined that the user has entered thesleeping state when the eyelids of the user are determined to be closedfor a specified duration of time.
 23. The control method according toclaim 20, wherein it is determined that the user has entered thesleeping state when the eyelids of the user are determined to be closedfor a specified duration of time.
 24. The device according to claim 1,wherein the at least three different power saving states are in additionto a normal display state of the device.
 25. The method according toclaim 19, wherein the at least three different power saving states arein addition to a normal display state of the device.
 26. The controlmethod according to claim 20, wherein the at least three different powersaving states are in addition to a normal display state of the imagedisplay unit.